High Performance Computer Architectures

Dennard scaling optimises by reducing size and voltage of transistors which allows us to increase frequency since the capacitance is reduced. Unfortunately, there’s a point where voltage is too low and there’s leakage making the transistors ineffective. This is known as the power wall. This is why CPUs have capped out around 5GHz. Dennard scaling drove Moore’s Law until we hit the power wall.

Computer performance increase far outpaces memory performance increase. Nowadays applications are typically bound by memory access rather than compute. You should be optimising for data movement.